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Abstract — Orthogonal Matching Pursuit (OMP) is a simple, 
yet empirically competitive algorithm for sparse recovery. Recent 
developments have shown that OMP guarantees exact recovery 
of A'-sparse signals in K iterations if the observation matrix $ 
satisfies the Restricted Isometry Property (RIP) with Restricted 
Isometry Constant (RIC) 5k+i < "T^qr^- On the other hand, 
OMP empirically promises higher recovery rates when it runs 
for more than K iterations. In order to support this theoretically, 
we extend the theoretical analysis of OMP to cover more than K 
iterations. We develop exact recovery guarantees for A-sparse 
signals in more than K iterations when $ satisfies an RIP con- 
dition which depends on the number of correct and false indices 
in the support estimates of intermediate iterations. In addition, 
we present an upper bound on the number of false indices in 
the support estimate for the derived RIP condition to be less 
restrictive than 5k+i < "T^qr^' Moreover, we provide recovery 
simulations which demonstrate the performance improvement 
when more than K iterations are allowed. Finally, we empirically 
analyse the number of false indices in the support estimate, which 
indicates that these do not violate the developed upper bound in 
practice. 

Index Terms — Compressed sensing, greedy algorithms, orthog- 
onal matching pursuit, restricted isometric property 



I. Introduction 

Sparse recovery problem aims at finding the i^-sparse 
^ ■ signal X e that satisfies a set of linear observations. 



Mathematically, this is expressed as 

X = argmin ||x||o s.t. y = $x, 



where * e R^'^><^\ y G R*^ and K <M < N. Problems of 
this or similar forms appear for signal recovery or approxima- 
tion in Compressed Sensing (CS) [r|-[4|, for finding sparse 
representations in overcomplete dictionaries ||5]-i2l and etc. 

Among others, Orthogonal Matching Pursuit (OMP) |6| is 
a canonical greedy algorithm for sparse recovery. It aims at 
finding the support, i.e. the set of nonzero indices, of x one by 
one. At each iteration, OMP identifies the index corresponding 
to the column of $ which has maximum correlation to the 
residue of y. Due to its simplicity and empirically competitive 
performance, OMP and its variants have been frequently used 
in sparse recovery and approximation problems IT], ll8l- lfT3l . 
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Research Council of Turkey (TUBITAK), Kocaeli, Turkey. 



A. Restricted Isometry Property 

Restricted isometry property (RIP) fT?] has been acknowl- 
edged as an important means for obtaining theoretical guaran- 
tees in recovery and approximation problems. RIP is defined 
as follows: 

Theorem 1 (Restricted Isometry Property): A matrix $ is 
said to satisfy the iiT-RIP if there exists a Restricted Isometry 
Constant (RIC) 5k satisfying Q < 5k < ^ and 

(1-<5k)||x||2<|1*x|1^<(1 + 5;,)||x||2, (1) 



for all X where ||x||o < K. A matrix that satisfies RIP acts 
almost like an orthonormal system for sparse linear combina- 
tions of its columns lfT4l . Random matrices with i.i.d. Gaussian 
or Bernoulli entries and matrices randomly selected from 
discrete Fourier transform were shown satisfy the RIP with 
high probabilities, when they satisfy some specific conditions 
on K, M and N flS^, lfT6l . 

RIP has been utilized for proving theoretical guarantees of 
exact recovery for many algorithms in CS literature. These 
include convex relaxation [T?!, [Tsl, fTTl and greedy algo- 
rithms such as Regularized OMP (ROMP) I.18J. Compressive 
Sampling Matching Pursuit (CoSaMP) |[T9l, Subspace Pursuit 
(SP) iWl, Iterative Hard Thresholding (IHT) f2V\ and etc. 

B. Recent Developments in Theoretical Analysis of OMP 

Initial contributions on theoretical analysis of OMP have 
concentrated on coherence |13| or probability analysis |9|. 
Just recently, RIP has been demonstrated to provide a straight- 
forward analysis of OMP. Davenport and Wakin have shown 
that exact recovery of any i^-sparse signal from noise-free 
measurements is guaranteed via OMP in K iterations when 
RIP is fulfilled with RIC satisfying 5k+i < [22 1. Lately, 
Wang and Shim have proven a less restricted bound for OMP 
1 23 1 which we visit in the following theorem: 

Theorem 2 (Exact recovery condition for OMP [23]): 
OMP perfectly recovers any if-sparse signal from noise-free 
measurements in K iterations if the observation matrix $ 
satisfies RIP iQU with 

Sk+1 < J (2) 



/K + 1 

The proof of Theorem |2] follows from induction [23 1. It can 
be shown that ^ guarantees the success of the first iteration. 
Then, this result can be generalized to all of the following 
iterations, guaranteeing exact recovery of any if -sparse signal 
in exactly K iterations. 
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C. Our Contributions 

Theorem 12] represents a very important finding for theoreti- 
cal analysis of OMP, yet it is restricted to the first K iterations. 
Empirically, OMP yields better recovery performance when 
it is allowed to run for more than K iterations, such as 
until the residue of the observation vector vanishes (see Il24l 
for instance). To get an intuitive understanding of this, let's 
consider $ satisfies /-RIP with some < < 1 where I is 
a bit larger than K. Then, selecting I indices in the support 
estimate does not actually harm the recovery, as soon as the 
correct support is a subset of the selected indices. On the 
contrary, this may cover for false indices introduced by the 
first K iterations. Hence, we intuitively expect better recovery 
performance when OMP is not restricted to K iterations. 

In this manuscript, we aim at closing this gap between 
empirical and theoretical analysis of OMP. For this purpose, 
we theoretically investigate OMP recovery from noise-free 
measurements without any restrictions on the number of 
iterations. In particular, we concentrate on the residue-based 
termination rule ||r'||2 < e||2/||2^ where r' denotes the residue 
of y after the Zth iteration. To avoid ambiguity, we use the 
term OMP^^ to indicate the restriction to K iterations, and 
OMPe for the residue-based termination rule. 

In contrast to OMPj^ , theoretical analysis of OMP^ presents 
a more complex case, since OMP^ may deal with erroneous 
iterations by its nature. Theoretical analysis is not limited to 
K iterations anymore, which interestingly turns out to be both 
the basis and the main obstacle of our analysis. Exploiting the 
fact that OMPg may choose more than K indices, we present 
in Section IIII-AI a sufficient RIP condition for the success 
of an intermediate iteration based on the number of correct 
(and false) indices in the support estimate of the previous 
step. Later, in Section IIII-BI we generalize this result for 
the success of all consequent iterations, and finally for the 
success of OMPg. Unfortunately, we cannot directly compare 
this condition to (|2]l, but we show that it turns out to be 
less restrictive than (|2|i when the support estimate contains a 
relatively large number of correct indices (or a small number 
of false indices). We use this result, together with the fact that 
OMPe enjoys all theoretical guarantees of OMPx, to state that 
OMPe recovers a wider range of sparse signals than OMP^-. 

On the other hand, turning our RIP condition into general 
exact recovery guarantees has not yet been possible because 
of the dependency on the number of correct and false indices 
in the support estimate. Unfortunately, we cannot provide any 
conditions to guarantee the existence of such support estimates 
in the intermediate steps. Therefore, we could not use our 
results to directly refine the general exact recovery guarantees 
of OMPe over From this point of view, limiting OMP 
with K iterations can be seen as a trade-off for providing 
general exact recovery guarantees. However, our theoretical 
analysis still reveal the improvements via OMPg over OMPa'. 
Moreover, the empirical phase transition curves, which we 
present Section |IV] for three different types of sparse signals, 
also support this theoretical findings, indicating that OMPg 
indeed encounters intermediate steps that contain enough num- 
ber of correct indices to satisfy the conditions we pronounce 



in this manuscript. In order to support this further, we also 
provide histograms of the number of unsuccessful indices 
after successful OMPe termination in Section IIV-BI These 
demonstrate that the number of unsuccessful iterations are 
empirically in the limits we present. 

II. Preliminaries 

A. Notation 

Let's first define the notation we use throughout this 
manuscript. T denotes the correct support of x. T' = 
{ti, ^2, ^i} is the support estimate for x after the l\h itera- 
tion of OMP, where ti is the index selected at the ith iteration. 
Tie and nf denote the number of correct and false indices 
in T\ respectively, i.e. |T n r'| = and |T - T'| = Uf. 
The observation matrix is decomposed as $ = [(f)i(l)2---4>N], 
where (pi is the ith column vector of ^. $t denotes the matrix 
consisting of the columns of # indexed by T, and x^ is the 
vector consisting of the elements of x indexed by T. Finally, 
r' is the residue after the orthogonal projection of y onto ^j^i 
by the end of the h\\ iteration. 

B. Preliminary Results 

The analysis we present in the next section is based on 
a number of preliminary results, which are discussed below. 
These include some observations that are well-known in the 
CS community as well as some results which we derive in 
this manuscript for our purposes. Specifically, Lemma [T] is a 
direct consequence of RIP, while Lemma |2] and Corollary [T] are 
from 1 20 1 and |19|, respectively. Corollary |2] is simply derived 
from Corollary [T] and Remark [T| is a direct consequence of 
Corollary |2] Finally, we derive Lemma |3] which we will later 
exploit for comparing the RIP bound of Theorem |2] with our 
result. The proofs are omitted either if they are very trivial, or 
they are already present in the corresponding references. 

Lemma 1 (Direct Consequence of RIP): Let 
/ C {1, 2, N}. For any arbitrary vector z e Rl^l 

(l-'5|/|)||z||2 < ||*1*/Z||2 < (l+5|,|)||z||2. 

Lemma 2 (Lemma 1 in fW^J): Let I, J C {1,2,...,A^} 
such that I f\ J = %. For any arbitrary vector z e RI'^I 

||*?*JZ||2 < ^|/| + |J|||z||2. 

Corollary 1 (Corollary 3.4 in 1119]): For every positive in- 
teger c and r 

5cr < cd2r- 

Corollary 2: For any positive integer K 



where \z~\ denotes the ceil of z, i.e. the smallest integer greater 
than or equal to z. 

Proof: Corollary |2] is a consequence of Corollary [T] We 
first replace c — 3 and r = \K/2] into Q. By rearranging 
terms, we get 

. S3[K/2] 
021K/2-] > ^ • 
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K +1 > 2\K/2\ hold by definition. Following monotonicity III. THEORETICAL ANALYSIS 

of RIC, we have 5k+i > S.^^m- Hence, we can write ^ ^^^^^^^ ^.^^^^ ^^^^^^.^^ ^^^^^^ 

Sk+1 > 52\K/2\ Having presented the necessary preliminary results, we can 

^z\K/2\ '^^^ move on to analysis OMPg. We start with success of a 

3 ■ single iteration, for which the theorem below states a sufficient 

condition depending on the number of correct and false indices 
in the support estimate. 

Theorems: Let IT' n T| = Ur and IT' - Tl = n, after 



> 



Remark 1 (Direct consequence of Corollary^: 
Theorem |2] is violated if 



/ 

iteration I. Then iteration ^ + 1 will be successful, i.e. e T, 
e ^ 3 if €> satisfies RIP with 

- TWTi- 1 

^K+rif+l < — = . (7) 

Proof: According to Corollary ID it is clear that ^ y/ K — ric + I 

contradicts Theorem l2] ■ n ^ \ i ■ ..u • t ^ 

' Proof: As r is the projection error of y onto ^^i, we 

Lemma 3: Assume K > 25. There exists at least one j^^^^ ^ Therefore (</)„rO = for all z e TK We can 
positive integer Uc < K that satisfies ^^^^ write 

„'\2 



^ <^==^— . (4) ||*TuT'r'||^ = E ('^-^' 



+ 1 VK -nc + l 
ic are bounded b 

8K + 4:Vk -4 



ieTUT' 

Moreover, such values of ?ic are bounded by — (0i,r')^, (8) 



ieT-T' 

K > ric > g . (5) Yvjjgj-g tijg righthand side of ^ contains only K — ric nonzero 

terms. Combining (O and the norm inequality, we obtain 



Proof: Set K — Uc ~ sK where < s < 1. Replacing s 

1 



into dUl, we get H^* /n 1 j 



*;,^^,r'||eo>^^=||*Wr'll2. (9) 
V-fi - ric 



< 



^/K + 1 y/sK + 1 Let X denote the estimate of x after iteration I. Then, r' can 

be written as 



Arranging the terms, we obtain the following bound for s: 



2 



r = y — ^j^iXrpi 



Then, the lower bound for is obtained as ^^^^^ ^ ^ ^^^^^^ j^^^^^j^ ^ Lemmaffl we obtain 

> ^K + ^VK-A^ > (l-5^+„^)|lz|U. (10) 



9 

On the other hand, ric < K requires sK — K — Uc > 1. 



Replacing ( fTOl l into (|9|l yields 



Hence, A' should satisfy ,., ^ 

ll*TUT'I"'l|co > ^=^||Z||2. (11) 

X > - 

s The selection rule for the index at iteration ( + 1 is 

/ 3Vk \ ' defined as 

- I 2 ] ■ = argmax|((/).,,r')|. (12) 

Rearranging terms we get Combining this definition with ( fTTT i. we obtain 

K>5Vk, = \\^*r'\\^ 



— II^TuT'^^'lloo 
1 — ^K+Uf . 



which is satisfied when K > 25. Combining this with we 

conclude (01) is satisfied if > — ' \\'^-\\2 

y/K - Uc 

%X + A\/K — 4 

K > nc> ^ Now, suppose that ^ T U T'. Then, we can write 

forii:>25. |(0ti+i,r')| = ||0t* ^^^TuT'^lb 

■ < '5A'+n/+l||z||2- 
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by Lemma |2l This never occurs if 



1-S 



K+n f 



\/K - Tic 



||z||2 > (5k+„, + i||z||2 



or equivalently 



(13) 



Following monotonicity of RIC, Sk+uj+i > ^K+jif Hence, 
(fTJl i is guaranteed when 



< 1 

which is equivalent to 



1 



(14) 



y/K - Tic + 1 

Hence, ti+i e T U when (O holds. We also know that 
((/)i,r') = for all i G TK Therefore, a selected index cannot 
be selected again in the following iterations, i.e. t^+i ^ T'. In 
combination with ( fT4b this directly leads to ij+i e T — T'. To 
conclude, given ric and Uf, ( fT4] i guarantees e T, that is 
iteration I + 1 will be successful. ■ 
Theorem [3] and Theorem |2] are in fact closely related. 
Theorem |2] is based on the fact that the RIP condition in 
^ guarantees exact recovery of an iteration, given that all 
previous iterations are successful. This is fair when we are 
interested in analyzing OMPk, however, analysis of OMPg 
should obviously allow unsuccessful iterations in addition to 
successful ones. Theorem [3] removes the dependency of the 
success condition on the success of all previous iterations, 
generalizing the success condition of a single iteration to a 
broader extend which can handle failures among previous 
iterations. The equivalency of these two conditions when no 
failures exist, i.e. nj — 0, can also be trivially shown. 

B. Recovery Analysis of OMP^ 

Theorem [3] can generalized to all of the following iterations 
until termination of OMR This leads to exact recovery once 
the conditions in Theorem[3]are satisfied. This is discussed in 
the following theorem: 

Theorem 4: Let \T^ C^T\ = and \T^ -T\ = Uf after 
iteration I. Then, OMPg perfectly recovers a JiT-sparse signal 
in I + Uf iterations if $ satisfies RIP with 

SK+nf+l < — ■ (15) 

y/K - Tic + 1 

Proof: We prove Theorem |4] by induction. According to 
Theorem [3] ( fTsT i already guarantees success of the iteration 
Z + 1. As a result of this, £ T and Tj+i will contain 
ric + 1 correct indices. Since the right hand side of (flST l 
increases monotonically with the number of correct indices in 
the support estimate, iteration / + 2 requires a less restrictive 
constraint on the RIC than iteration I + 1 does. Therefore, 
(flST l also guarantees success of the iteration I + 2. Moreover, 
this is also valid for all of the following iterations, as each 
will require a weaker constraint on RIC. Hence, after iteration 
I + Uf, the support estimate Ti^nf will contain K correct 
indices, i.e. T e 7]+„j,. (fTsT i finally guarantees that the 



orthogonal projection coefficients of y onto T;+„^ yield x. 

■ 

Being an extension of Theorem [3] Theorem |4] also depends 
on Uc and Uf. This allows us to prove that OMPg recovery 
will be successful once the number of correct indices in an 
intermediate step / is high enough to satisfy (flST l. Yet, it 
also avoids us to generalize the results to complete recovery 
guarantees independent of ric and rif, as the existence of 
intermediate steps with enough number of correct indices (or 
a small number of false indices) is hard to guarantee. We 
cannot provide a proof of this for the time being, however, 
the histograms of Uf which we present in Section IIV-BI 
demonstrate that Uf is indeed bounded in practice. In addition, 
the empirical phase transitions in Section IIV-AI state that 
OMPe provides higher recovery rates than OMPk- These re- 
sults indicate that OMPg recovers some signals which OMP/<- 
cannot, as OMPg encounters intermediate steps with enough 
number of correct indices to satisfy (flST l in practice. 

C. On the Theoretical Comparison of OMP^ and OMPk 

Instead of directly improving the exact recovery guarantees 
for OMPe over Theorem |2] we follow a different strategy, 
and show that OMP^ perfectly recovers not only all signals 
which OMPk can recover, but also some others which OMPk 
cannot. To achieve this result, we provide a comparison of 
the RIP conditions in Theorem |4] and Theorem |2] below. We, 
first, prove that Theorem |4] requires a less restrictive bound 
on the RIC than Theorem |2] does when Uc is high, and n j 
is small. Next, we combine this result with the fact that all 
exact recovery guarantees for OMP^j- are also valid for OMPe 
to show the superiority of OMPe recovery. 

In order to state that (fTSl l. at least for some particular 
cases, presents a less restrictive constraint than (|2]i, we need 
to compare the two bounds: 

1 „ 1 



/K+1 -■-J" y/K-Uc + l 

Unfortunately, that right and left-hand sides of the two bounds 
are related in the same direction makes dkect comparison 
impossible: 



OK+rif + l 
1 



> 



> 



''K+1, 
1 



VK -He + 1 y/K + 1 

Intuitively, when Uf is small, and ric is large, we expect 
Theorem|4]to be less restrictive. To illustrate, consider n/ = 1 
and Uc ^ Uf. In this case. Theorem |4] requires an RIP con- 
dition based on Sk+2 instead of Sk+i of Theorem |2l i.e. two 
RJC's are practically close to each other. However, the bound 
in ( fTSl ) is significantly larger than ^ because of Uc being 
large. Hence, ( fTsT i becomes practically less restrictive in this 
situation. On the other hand, exact mathematical comparison 
of these two conditions is tricky, as it is not easy to obtain a 
tight bound on 6K+nf+i/SK+i for all n/. However, even by 
employing a loose bound on SK+uf+i/^K+i, we may state 
that ( fTSl ) is less restrictive than Q for some sparsity range: 

Theorem 5: Assume that K > 25, 1 < rif < \K/2] and ric 
satisfies (|5]l at iteration I. Then, (flST l becomes less restrictive 
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than (|2]) at iteration /. Hence, it is possible to satisfy (fTST i 
though (|2|i does not hold. 
Proof: Assume that 

3 



K+l 



(16) 



Since n/ < \K/2\, we observe that 3[i^/2] > K + nf + \. 
Following monotonicity of RIC, we obtain 

Remark [T] guarantees failure of (|2|i for this case. (This ac- 
complies with the OMP^ failure following the assumption 
Uf > 1.) 

On the other hand. Lemma |3] leads to 

3 1 
— < 



+ 1 \/K-nc + 1 
when Q is satisfied and K > 25. Hence, there exists some 
SK+uf+i such that 

S OK+nj + l 



< 



To conclude, when the parameters K, Uf and Uc satisfy 
the assumptions, there exists some Sx+uf+i which fulfill 
dTsT l, though (|2]does not hold for Sk+i- Hence, (fTSl l is less 
restrictive than ^ for this range of parameters. ■ 

To explain Theorem|5l let's assume that ^ is violated, and, 
moreover, OMP^ fails. (Note that violation of dU does not 
necessarily result in OMPk failure, but vice versa.) According 
to Theorem |5] it is still possible to satisfy (fTsT i in case rif is 
limited and is large enough to satisfy (|5]). This leads directly 
to the following remark: 

Remark 2: Assume Theorem |2] is violated, and OMPa- 
fails. If K, Uf and ric satisfy the assumptions in Theorem |5] at 
any intermediate iteration. Theorem |4] becomes less restrictive 
than Theorem |2] and may still be fulfilled. Hence, there exists 
a range of sparse signals which OMPg can perfectly recover, 
while OMPk cannot. 

Next, we state the following trivial remark: 

Remark 3: All exact recovery guarantees for OMP^ are 
also valid for OMPe,as the first K iterations of OMPg and 
OMPx are identical. Furthermore, given noise-free observa- 
tions, any particular signal exactly recovered by OMPa' is also 
exactly recovered by OMPg. 

Combining Remark |2] and Remark |3] we conclude that 
OMPe not only exactly recovers all instances OMP^r does, 
but also those where false indices in the support estimate stay 
limited. This clarifies that OMPe provides perfect recovery 
for a wider range of sparse signals, and hence higher exact 
recovery rates than OMPa' does. 

Note that the assumptions K < 25 and Q in Theorem |5] 
rely on the upper bound in rif < \K/2], which is chosen so 
that ( [TtI i can be established. In other words, both K < 25 and 
(|5]l actually apply for the boundary condition Uf = \K/2'] — 
1. This condition is sufficient to prove Theorem |5] however, 
it is not really necessary in practice. In practice. Theorem |5] 
should also hold for smaller K and ric values when nj is 
also small. Hence, we expect OMPg to perform better than 



OMPa not only for K < 25, but also for smaller K values. 
Moreover, the lower bound on Uc will also be lower than Q 
when n j is small. Unfortunately, extending Theorem |5] for 
smaller K values requires a tighter bound on (5A+n/+i/'^A'+i 
for smaller Uf values, which we are not able to provide yet. 
However, empirical recovery analysis in the next section state 
that OMPe improves recovery even when K < 25. Moreover, 
the histograms of rif in Section HV-BI indicate that the bound 
Uf < \K/2'] is usually loose in practice. 



IV. Empirical Analysis 

A. Phase Transitions 

In this section, we support our theoretic results by compar- 
ing empirical recovery performances of OMPg and OMPa- 
For comparison with other algorithms, we also include Basis 
Pursuit (BP) [T] and SP. We run the simulations for three 
different nonzero element distributions. The nonzero elements 
of the 'so-called' Gaussian sparse signals are drawn from the 
standard Gaussian distribution, while those of the uniform 
sparse signals are distributed uniformly in [—1,1]. The last 
ensemble involved is the Constant Amplitude Random Sign 
(CARS) sparse signals (following the definition in |4|) where 
nonzero elements have unit magnitude with random signs. 
For OMPe, e = 10^^ and maximum iterations are limited 
by AI. The exact recovery condition for x is specified as 
||x — x||2 < 10^^||x||2 where x is the reconstructed sparse 
vector. 

We compute the empirical phase transitions in order to 
provide an extensive evaluation over a wide range of K 
and M. Let's define normalized measure for the number of 
observations as A = M/N and for sparsity level as p = K/M. 
We keep = 250 fixed, and alter M and K to sample 
the {\,p} space for A G [0.1,0.9] and p e [0,1]. We 
randomly generate 200 sparse instances for each {A, p} tuple. 
Next, we draw a random Gaussian observation matrix for 
each test instance and run each algorithm to recover x. After 
recovery of all samples, we compute the phase transitions by 
the methodology described in |4|. This methodology uses a 
generalized linear model with logistic link to describe the exact 
recovery curve over p for each A. Then, the phase transition 
curve is finally given by combining the p values which provide 
50% exact recovery rate for each A. 

Fig. IIV-AI depicts the phase transition curves of OMPe, 
OMPa, bp and SP for Gaussian, uniform and CARS sparse 
signals. For all distributions, OMPe yields better phase transi- 
tion over OMPa, which is parallel to our theoretical findings. 
On the other hand, recovery performance of OMP highly 
depends on the coefficient distribution, while BP is robust 
to coefficient distribution, and SP shows less variation than 
OMP. At one end stands the Gaussian sparse signals, where 
OMPe outperforms BP and SP. For the uniform sparse signals, 
OMP might also be considered as the most optimal algorithm 
over the whole A range. In contradiction, CARS sparse signals 
are the most challenging ones for OMP, whose performance 
severely degrades for this ensemble. 
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Fig. 1. Empirical phase transitions of OMPe, OMPk, BP and SP for recovery of Gaussian, uniform and CARS sparse signals from noise-free observations. 
The entries of observation matrices are selected as i.i.d. Gaussian random variables. The results are obtained over 200 trials. The axes labels are defined as 
p = K/M and A = M/N where N = 250. 
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Fig. 2. Histogram of failed OMPe iterations (n/) over 200 Gaussian sparse vectors, nf = corresponds to the samples which are successfully recovered 
by OMPa'. OMPa' perfectly recovers 1 19 out of 200 samples when M = 125, N = 40 and 97 out of 200 samples for M = 150, = 52. OMP^ recovers 
all samples perfectly in both cases. The number of failed OMPe iterations do not exceed K/A for both cases. 



B. Empirical Success and Failure Rates on OMPp Iterations 

Theorem |5] is based on the assumption nj < \K/2\, which 
leads to the other constraints on K and ric, i.e. K <2b and (|5]l. 
Hence, satisfying the limit on the number of failed iterations 
is critical for Theorem |5] On the other hand, the bound 
Uf < \K/2\ may also be loose for many practical examples, 
making these constraints too strong in practice. Therefore, it is 
worthwhile to investigate the number of failed iterations after 
the termination in order to validate these assumptions. 

For this purpose, we choose two of the previous tests, and 
depict the histograms of Uf after successful termination of 
OMPe. Successful termination is important here, as OMPe 
may run until it reaches the maximum number of iterations 
(M) in case of a failure. Therefore, we consider two cases 
where OMPe perfectly recovers all instances, while OMP^ 
cannot, namely M = 125, if = 40 and M = 150, /f = 52. 
The histograms of failed iterations are depicted in Figure 
OMPx recovers 119 out of 200 instances perfectly for the 
first case, and 97 for the latter These correspond to the region 
where no OMPe iterations fail, i.e. ri/ = in the plots. In 



accordance with our theoretical findings, OMPe takes some 
wrong steps to reach perfect recovery, when OMPjc fails. We 
observe that the number of these steps is smaller than the 
upper bound \K/2\ — 1. Actually, in both tests OMPe never 
takes more than K/A wrong steps. Hence, the assumption 
Uf < \K/2\ turns out to be empirically loose for these two 
cases. 

V. Conclusion 

In this manuscript, we discuss theoretical and empirical 
analysis which provide a better understanding of OMP recov- 
ery performance with termination criteria based on the residual 
power under noise-free observations assumption. This type of 
termination criterion presents a more optimal objective than 
setting the number of iterations equal to K when the aim is 
finding an exact if-sparse representation, rather than obtaining 
a ii'-sparse approximation. 

Theoretical analyses in Section |III] state an exact recov- 
ery condition for OMPe based on ric and Uf after some 
intermediate iteration. Though we cannot cast this into an 
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improved general RIP bound independent of Uc and n/, we 
still State that this condition is less restrictive than the currently 
known best RIP bound for OMP/^ when ric and n/ satisfy 
some assumptions. In combination with the validity of all 
theoretical OMPx guarantees for OMPg, this analysis explains 
the improved performance of OMPe for recovery of sparse 
signals from noise-free observations. 

We finally test the validity of our theoretical analysis via 
recovery simulations involving sparse signals with different 
nonzero coefficient distributions. The phase transitions in Sec- 
tion IIV-AI reveal that OMPe empirically outperforms OMPk 
for all distributions involved. Moreover, these also yield that 
OMPe is capable of providing better recovery rates than BP 
and SP when the nonzero elements follow Gaussian or uniform 
distributions. In addition, we present the histograms of Uf 
in order to show validity of the assumption Uf < \K/2]. 
These histograms indicate that our assumptions are not only 
reasonable, but also loose in practice. 

To conclude, we address an important issue for theoretical 
analysis of OMR The theoretical results presented in this 
manuscript reveal the effects of the number of iterations on the 
recovery performance of OMR We feel sure that our findings 
will provide a basis for improving general exact recovery 
guarantees of OMR as part of future work in the field. 
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